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Fall 2015 Regional PBIS School Status and Training Updates 


Summary of results from school training and implementation fidelity measures 
October 2015 


This Fall 2015 school status and training report provides a summary of the current implementation status 

of schools in training results from the Fall 2015 training surveys. The Minnesota Department of Education 
(MDE) and Regional Training Partners (RIPs) sponsor three training sessions per year for schools that are 
participating in Minnesota’s 2-year training sequence for Positive Behavioral Interventions and Supports 
(PBIS). This report is intended for all stakeholders to better understand the status of PBIS of schools in training 
and help to refine training and data reporting. More information about PBIS in Minnesota can be found at: 


www.pbismn.org. 


This report summarizes the results of the Fall 2015 training surveys. 
=" There are a total of 52 Cohort 10 schools that are in their second year of training. 
=" There are a total of 56 Cohort 11 schools are in their first year of training. 


Wilder Research was contracted to evaluate the PBIS initiative statewide. As a part of the PBIS evaluation, 
surveys are conducted with school teams who are participating in the training sequence (one survey was 
completed per school team). The results of these surveys, attendance records from the RIPs, data system 
information gathered from the RIPs and from school teams, and data entered for the TIC (Cohort 10 only), TFI 
(Cohort 11 only), and SAS (both Cohort 10 and 11) are the basis of this report. 


Training attendance 


Schools in their first year of training (Cohort 11 in 2015-16) receive two days of training at each of three 
points during the year, and schools in their second year of training (Cohort 10 in 2015-16) receive one day 
at each of three points during the year. Overall, attendance for the Fall 2015 trainings was very high. 


All school teams in Cohort 11 attended fall training this year. There is one metro school from Cohort 10 that did 
not attend and will be receiving make-up training this fall. One school from the north in Cohort 10 is no longer 
participating in PBIS. Overall, most team administrators from both cohorts attended training. See Figure 1. 
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1. Fall 2015 training attendance by Cohort (number of school teams / administrators attended) 


Cohort 10 

Metro North South 
team - full 20 18 13 
team - partial 0 0 0 
team - not attended 1 0 0 
administrator - full 18 16 13 
administrator - partial 0 0 0 
administrator - not attended 3 2 0 
Total number of teams 21 18 13 

Cohort 11, day 1 Cohort 11, day 2 

Metro North South Metro North South 
team - full 19 18 19 team — full 19 18 19 
team - partial 0 0 0 team — partial 0 0 0 
team - not attended 0 0 0 team - not attended 0 0 0 
administrator - full 17 18 19 administrator — full 17 17 19 
administrator - partial 0 0 0 administrator — partial 0 0 0 
administrator - not attended 2 0 0 administrator - not attended 2 1 0 
Total number of teams 19 18 19 Total number of teams 19 18 19 


Overall satisfaction with training 


School teams were asked to indicate how much they agree or disagree that the training was a positive, 
worthwhile experience overall. The vast majority of school teams indicated satisfaction with the overall 
training experience. However, a few schools in Cohort 10 from each region indicated some dissatisfaction 
with their training experience. See Figure 2. 


2. School team ratings: “Overall, this training was a positive, worthwhile learning experience.” 


Cohort 10 Cohort 11 
Metro North South Metro North South 
Strongly agree 44% 47% 14% 63% 64% 46% 
Agree 50% 47% 79% | 37% 36% 54% 
Disagree 6% 1% 1% 0% 0% 0% 
Strongly disagree 0% 0% 0% 0% 0% 0% 


Note: Combined percentages of the “strongly agree” and “agree” categories that equal 85% and above are highlighted in green 
indicating a high level of satisfaction. Combined percentages of the “disagree” and “strongly disagree” categories that equal 15% 
and above are highlighted in orange to indicate a relatively lower level of satisfaction (none on this table). 
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Ratings of specific training content areas 
Cohort 10 


School teams were asked to rate how useful the specific training content areas is in terms of helping them 
implement PBIS at their school. For Cohort 10 (Year 2), “Action planning (Metro and North regions 
only),” “Tier 2: Managing Escalating Behavior,” and “Function-based thinking (North and South regions 
only)” are the sessions that were most likely to be viewed as vitally or very useful. The “Re-establishing 
Tier 1 teams,” “Re-establishing buy-in and using data (South region only),” and the “Update TIC” sessions 
were most frequently rated as “somewhat useful” by the majority of school teams. See Figure 3. 


3. Ratings of specific training content areas —- Cohort 10 (Year 2) 


ODR Data 
Update TIC 


Action Planning 


Re-establish buy-in/ 
Using data 


Tier 2: Managing 
escalating behavior 


Tier 2: Interventions & 
selection based on data 


Tier 2: Overview of 
critical features 


Fidelity Data: Instruments 
and assessments (BoQ, take 
the TIC, action planning) 


Function-based thinking 
(ex. FACTS) 


Re-establishing Tier 1 teams 
(take the TIC and review 
steps for Action Planning) 


0% 20% 40% 60% 80% 100% 
= Vitally useful m= Very useful Somewhat useful sm Not at all useful 
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Additionally, schools from Cohort 10 were asked in an open-ended format what parts of training were the 
most and least useful to their PBIS teams. The overwhelming majority of school teams in all regions stated 
that planning and work time were the most useful parts of their training. Teams from the Metro and North 
regions noted that the Escalating Behavior training session was particularly useful to them. Below are a few 
of their comments, edited for clarity: 


‘Action planning. Getting ready for the beginning of the year and rolling it out school wide [was the most 
useful for us].” —Cohort 10 team, Metro region 


‘The managing escalations piece. A great piece to share with our new staff, new teachers, and our veteran 
teachers.” —Cohort 10 team, North region 


School teams from Cohort 10 also provided feedback on the least useful parts of fall training. Teams had 
varying responses both within and across regions as to what the least useful part of training was. Below are a 
few of their comments, edited for clarity: 


“The new TFI information. It was not presented well. | feel like it will be good, but for now it’s not a useful piece.” 
—Cohort 10 team, Metro region 


“Updating the TlIC—that could be preparation for training to get schools back in the [PBIS] mindset.” 
—Cohort 10 team, North region 


Cohort 11 


For Cohort 11 (Year 1), “Overview of the 8-step Planning Guide,” “Establishing Leadership Team/Buy-in,” 
“Mission and Behavior Purpose,” and “Identifying 3-5 Expectations (Metro and South regions only)” were 
most likely to be rated as vitally or very useful by teams in all regions for the first day of training. For the 
second day of training, the sessions on “TFI,” and “Time to work on Steps 3 and 4,” were the most highly 
rated. On the other hand, the sessions “Working Smarter (App B)” and “Intro to Action Planning and 
Template (North only)” were rated the least useful; however, each still had a majority citing them as 
somewhat or very useful. See Figure 4. 
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4. Ratings of specific training content areas - Cohort 11 (Year 1): Day 1 


Identify 3-5 Expectations 


How to Get and 
Maintain Buy-in 


Establishing Leadership 
Team/Buy-In 


Overview of the 8-step 
planning guide 


Getting Started: 
Implementation of Primary 
Prevention 


PBIS Rationale, Definitions, 
and Features 


0% 20% 40% 60% 80% 100% 
= Vitally useful m= Very useful Somewhat useful sm Not At All Useful 
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Tiered Fidelity Inventory (TFI) North 


Basic Overview of Outcome, North 


Ratings of specific training content areas — Cohort 11 (Year 1): Day 2 
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Cohort 11 schools were also asked in an open-ended question format to identify the most and least useful 
training components. Similar to Cohort 10, the vast majority of school teams in Cohort 11 commented that 
the time to work together in teams was the most useful part of training. See below for a few of their 
comments, edited for clarity: 


“The work time was essential and we could have used more. Also, the coaches were beneficial when we had 
questions.” —Cohort 11 Team, Metro region 


“The team work time following explanations was very convenient for our team. Developing an action plan and being able to 
come away with tangible items such as a purpose statement and behavior rubrics was very useful for our team. It was nice 
there were multiple trainers that were able to help our team when we had questions.” —Cohort 11 Team, South region 


Additionally, Cohort 11 school teams provided feedback on what they found to be the least useful training 
components. Similar to Cohort 10, answers varied greatly across teams and regions. A few schools from the South 
and North region felt the TFI was not useful because it was too early. See below for some of their comments: 


“The information on TFI and data collection was a little ahead of where we are at currently. The information provided a 
good reference for the direction we are moving, but it was not useful for us at this time.’—Cohort 11 team, South region 


“We felt rushed with work time and presentations. We needed more work time.’—Cohort 11 team, Metro region 
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Ratings of training effectiveness 


School teams were also asked how strongly they agree or disagree with five statements about the 
effectiveness of the trainings. A majority of all school teams in all regions reported that the training will 
improve practices in their school, and that their team is confident to meet with school staff and share topics 
learned at the training. Some school teams from Cohort 11 in the South region disagreed that the training 
information was clearly presented. A few teams from Cohort 10 in the Metro and South region disagreed 
that the training enhanced their understanding of PBIS. Similarly, about one-quarter of school teams from the 
Metro and South regions in Cohort 11 disagreed that the resources, materials, and teaching aids providing in 
training were helpful. See Figure 5. 


5. Ratings of training effectiveness 


Cohort 10 Cohort 11 

Metro North South Metro North South 
The training information was clearly presented. 
Strongly agree 33% 20% 21% 42% 36% 23% 
Agree 67% 80% 1% 53% 64% 62% 
Disagree 0% 0% 7% 5% 0% | 15%) 
Strongly disagree 0% 0% 0% 0% 0% | Oh 
The training enhanced our understanding of PBIS. 
Strongly agree 44% 40% 21% 58% 64% 54% 
Agree 39% 47% 50% 37% 36% 31% 


Strongly disagree 0% 0% a 0% 0% 


The training will improve our practices in school. 


Strongly agree 39% 40% 21% 63% 64% 62% 
Agree 56% 47% 64% 37% 36% 38% 
Disagree 6% 13% 14% 0% 0% 0% 
Strongly disagree 0% 0% 0% 0% 0% 0% 

The resources, materials, and teaching aids provided in training were helpful. 

Strongly agree 22% 33% 21% 26% 21% 23% 
Agree 72% 60% 1% 47% 79% 54% 
Disagree 6% 7% 7% | 26 0% SBR 
Strongly disagree 0% 0% 0% | 0% 0% | 0% 
Our team is confident in our ability to meet with school staff and share topics learned today. 

Strongly agree 39% 47% 21% 47% 50% 69% 
Agree 61% 53% 1% | 47% 36% 31% 
Disagree 0% 0% T% | 5% 14% 0% 
Strongly disagree 0% 0% 0% | 0% 0% 0% 


Note: Combined percentages of the “strongly agree” and “agree” categories that equal 85% and above are highlighted in green 
indicating a high level of satisfaction. Combined percentages of the “disagree” and “strongly disagree” categories that equal 15% 
and above are highlighted in orange to indicate a relatively lower level of satisfaction. 
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Completion of PBIS activities 


School teams were asked if they had completed key PBIS activities at training or within the past month. A 
majority of school teams in all regions partially completed their action plan, except the north region where 
half of school teams completed it and the other half did not. The majority of teams from Cohort 10 in the 
South and North regions reviewed their Office Discipline Referral (ODR) data in the past month. Almost 
two-thirds (61%) of teams from the metro region in Cohort 10 did not review their ODR data. The majority of 
school teams in all regions from Cohort 11 reported they did not review ODR data at within the past month. 
Use of data for planning and making decisions is a key element of training, so it would be expected that 
schools entering their second year should be internalizing and implementing this concept while schools at 
their first training might just be establishing data systems and are just beginning to use ODR data. 


Schools were also asked to what extent they have researched or implemented Tier 2 and Tier 3 interventions 
at their school within the past month. The majority of schools in Cohort 11 did not research or implement 
either Tier 2 or Tier 3 interventions. At least half of the schools in all regions in Cohort 10 reported that they 
researched Tier 2 interventions. Nearly half of schools from the north region in Cohort 10 report that they 
have “fully” or “partially” completed implementation of Tier 2 interventions. The majority of schools in all 
regions for Cohort 10 note that they have not researched or implemented Tier 3 interventions. See Figure 6. 


6. Completion of PBIS activities at training or within the past month 


rr Cohort 10 Cohort 11 

Activity 
Metro North South Metro North South 

Update your school’s PBIS Action plan 

Fully completed 50% 33% 29% | 0% 0% 15% 

Partially completed 39% 67% 50% | TA% 50% 62% 
Review your school’s Office Discipline Referral (ODR) Big 5 data 

Fully completed 22% 47% 21% | 11% T% 8% 

Partially completed 17% 33% 36% | 21% 14% 15% 


Not completed 6% 20% 48% | 68H 78% TT 


Taken a fidelity assessment (TIC, SAS, SET, TFI, etc.) 
Fully completed 12% 60% 1% 63% 57% 85% 
Partially completed 11% 27% 1% 32% 21% 8% 
Researched Tier 2 Interventions 
Fully completed 11% 13% 0% 0% 0% 8% 
Partially completed 39% 33% 43% 5% 0% 0% 
Implemented Tier 2 Interventions 
Fully completed 11% 7% 0% | 0% 0% 8% 
Partially completed 17% 40% 21% | 5% 0% 0% 


Not completed TR BB 79% — | 95% 100% «92% 
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6. Completion of PBIS activities at training or within the past month (continued) 


fo Cohort 10 Cohort 11 
Activity 
Metro North South Metro North South 
Researched Tier 3 Interventions 
Fully completed 6% T% 0% | 0% 0% 8% 
Partially completed 11% 40% 36% | 5% 0% 0% 


Not completed 8% 53% 64% | 95H 100% 92% 


Implemented Tier 3 Interventions 
Fully completed 6% 7% 0% | 0% 0% 8% 
Partially completed 6% 33% 14% | 5% 0% 0% 


Not completed 89% 60% 86% | 95% 100% 92% 


Note: Cells in the ‘fully completed” category that are 80% and above are highlighted in green to indicate a high level of activity completion. 
Cells in the “partially completed” category that are 50% and above are highlighted in blue to indicate a high level of partial completion. 
Cells in the “not completed” category that are 10% and above are highlighted in orange to indicate a low level of activity completion. 


School teams were asked in the open-ended format to identify any Tier 2 or Tier 3 interventions that they 
have either researched or implemented. Cohort 10 school teams named a variety of Tier 2 interventions 
that they had researched or implemented. The most common intervention either researched or implemented 
at this level was check-in/check-out (n=7 teams across all regions). Some other Tier 2 interventions named 
were Work Zone Area, Boys Town Social Thinking, IEP, Buddy Rooms, Behavior Contracts, Red Tag, 
and Explicit Modeling. Very few Cohort 10 teams report implementing or researching Tier 3 interventions. 
The Tier 3 interventions specified were: Boys Town, SAIL, Level 3 EBD, Individual Counseling, STAT, 
expulsion, and alternative school referrals. One metro school from Cohort 11 indicated that they were 
doing research around Tier 3 IEPs. The rest of the schools reported that they had not researched or 
implemented any Tier 2 or Tier 3 interventions. 


Strengths and challenges implementing PBIS 


Schools from both cohorts in all regions were asked in an open-ended format to identify what successes 
and challenges they are facing with PBIS implementation. 


Successes 


Cohort 10 school teams across all regions commonly identified Student and Staff Buy-in (n=14), Rewards 
Systems (n=12), and Teaming (n=9) as the areas they had the most success with PBIS implementation. Other 
less-common themes included Establishing Expectations (n=6) and Data Use (n=3). Below are some of their 
comments, edited for clarity: 


“Buy-in. Teachers are excited about PBIS and growing it at our school.’—Cohort 10 team, Metro region 


“We think we are having good success with rewarding our students both individually and at the classroom grade 
level.’—Cohort 10 team, North region 
“We have people on our team that are ‘doers’ and we have good communication. We work well as a team and our 
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staff all works well together.’---Cohort 10 team, South region 


“Monthly assemblies and overall student buy-in has been very successful. Recognizing and promoting behavior 
expectations caught on very quickly and it is used consistently by staff and students.” 
—Cohort 10 team, North region 
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School teams in Cohort 11 from all regions identified Teaming (n=10) as the most successful component 
of their PBIS program. Some schools in all regions reported that Establishing Expectations (n=7) to be a 
successful program element. A few teams in the Metro and North regions identified the Behavior Matrix (n=6) 
as a success. A smaller number (n=4) teams from the Metro and South regions indicated that Buy-In was a 
successful part of their PBIS program. Many (n=14) Cohort 11 teams in all regions reported that they were 
“Just starting” and felt they could not identify the most successful elements of their PBIS initiatives yet. 
Below are some of their comments, edited for clarity: 


“[We have had success] developing common goals within our team and being on the same page with each 
other.’—Cohort 11 team, Metro region 


“We have developed expectations in common areas with student-friendly pictures and language.” 
--Cohort 11 team, Metro region 


“The PBIS team is energetic and excited and keeps moving forward. [We have also] successfully completed the 
matrix and referral [system].’—Cohort 11 team, North region 


“The creation of our Behavioral Purpose Statement and our Behavior Expectation Matrix has been successful.” 
—Cohort 11 team, North region 


“Developing our team and achieving staff buy-in [have been successful for us]."—Cohort 11 team, South region 


Challenges 


Cohort 10 schools in all regions identified Staff Buy-in (n=15) as the most common challenge their teams 
faced with PBIS implementation. The North and South regions identified Time and Money (n=8) as a 
barrier, and the Metro and South regions said Using Data (n=5) was a challenge for PBIS implementation 
in their schools. Four school teams from the Metro region also identified moving to other tiers (beyond 
Tier 1) as a challenge. Below are some of their comments, edited for clarity: 


“Maintaining buy-in and excitement among staff [and presenting that PBIS] is not just another passing initiative. 
Convincing staff for the need of consistency school-wide [has been a challenge]."—Cohort 10 team, Metro region 


‘Time to train teachers and staff while maintaining a balance with our other initiatives and goals [has been a 
challenge].’—Cohort 10 team, Metro region 


*Funding for rewards/prizes, and time to incorporate and plan out all of our awesome ideas [has been a 
challenge].’—Cohort 10 team, North region 


‘Data collection and progress monitoring [have been a challenge].’"—Cohort 10 team, Metro region 


Many Cohort 11 schools across all regions identified the biggest challenge in implementing PBIS at their 
school to be Staff Buy-in (n=18) and some schools in the Metro and North regions (n=8) named 
consistency in rolling out the PBIS program among school staff to be their biggest challenge. Other 
challenges named in these two regions were time and money/funding (n=6). One other less-commonly 
mentioned theme in two of three regions was Using Data (n=3). See below for a few of their comments, 
edited for clarity: 


‘Although we feel that staff buy-in is a strength, we still struggle to get buy-in from the 10-20 percent who may 
oppose the philosophical change.’—Cohort 11 team, South region 


“(We struggle with] consistency among ALL staff members, we need more buy-in as well as more ideas as to how 
[to implement PBIS].’—Cohort 11 team, Metro region 
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Implementation fidelity measures 


PBIS is an evidence-based approach to addressing behavior issues in schools. A significant amount of 
research has been done to identify the critical features of PBIS. More generally, implementation science 
points to a specific sequence to ensure the model is implemented with fidelity. The following assessment 
tools are used to assess PBIS implementation fidelity among participating schools in Minnesota: TIC, 
SAS, SET, TFI (Cohort 11 only), and the BoQ. The BoQ is not used with schools that are in training. The 
Tiered Fidelity Inventory (TFD) is a new instrument that measures implementation fidelity on all three tiers of 
PBIS and effectively replaces the TIC and the SET. It was rolled out for use in Minnesota schools in the Spring 
of 2015. Currently, only Cohort 11 is using this assessment. Cohort 10 will still utilize the TIC and the SET 
to assess their implementation fidelity, but will be given the option to use the TFI once they have 
completed training. The TIC, SAS, and TFI scores from August-October 2015 are reported here. 


Team Implementation Checklist 


The Team Implementation Checklist, or TIC, is a tool used by schools to measure PBIS implementation 
fidelity. The TIC is designed to be completed by the PBIS Team three times per year to monitor activities 
for implementation of PBIS in a school. In order to be on track with MDE’s data calendar, Cohort 10 schools 
in training should complete at least one TIC by the end of August and another one by the end of November. 
(School teams are typically instructed to complete the TIC at the August and November PBIS training 
sessions. ). The number of TICs completed this fall from Cohort 10 is included in the graph below (Figure 7). 


7. Number of TICs completed during fall 2015 


Cohort 10 (N=52) OTICs 1 TIC 2 TICs 
Metro (N=21) 2 19 0 
North (N=18) 2 16 0 
South (N=13) 1 12 0 
Total for cohort 10 5 47 0 


Strengths and challenges of the TIC 


School teams in Cohort 10 completed the Team Implementation Checklist (TIC) at training and were 
asked to identify areas of strengths and challenges based on their TIC outcomes. Most teams in all regions 
reported “Establish commitment” and “Establish and maintain team” to be a strength. Many schools in all 
regions also report “Self-assessment” to be a strength. The majority of schools in the Metro region state 
that establishing prevention systems is a strength while over one-quarter of teams in the North and South 
regions believe this to be a challenge. Forty percent of school teams in the north region report classroom 
behavior support systems to be a challenge and some teams from the Metro and South regions report that 
they “have not started” work on this component. Nearly three-quarters of schools from the North region 
report establishing an information system to be a challenge. One in ten schools from the Metro region and 
two-fifths of schools in the South region state they “have not started” to establish an information system. 
Over half of schools from the North region find “Building Capacity for Function-Based Support” to be a 
challenge. One-third of schools in the Metro region and slightly more schools in the South region note that 
they have not started this component. See Figure 8. 
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8. Strengths and challenges of the TIC—Cohort 10 


Cohort 10 
Activity Metro North South 
Establish commitment 
It's a strength 94% 87% 57% 
It's a challenge 6% 13% 36% 
Have not started 0% 0% 1% 
Establish and maintain team 
It's a strength 78% 93% 86% 
It's a challenge 22% 7% T% 
Have not started 0% 0% 1% 
Self-assessment 
It's a strength 18% 67% 1% 
It's a challenge 22% 33% 14% 
Have not started 0% 0% 1% 
Establish school-wide expectations: prevention systems 
It's a strength 89% 73% 64% 
It's a challenge 11% 27% 29% 
Have not started 0% 0% 1% 
Classroom behavior support systems 
It's a strength 61% 53% 57% 
It's a challenge 17% 40% 29% 


Establish information system 
It's a strength 50% 
It's a challenge 39% 
Have not started 
Build capacity for function-based support 
It's a strength 33% 
It's a challenge 33% 


27% 
13% 
0% 


33% 


53% 


Haverot started [88 18% 


50% 
29% 


29% 
29% 


Note: Cells in the ‘it’s a strength” category that are 80% and above are highlighted in green to indicate an area of strength. Cells 
in the “it’s a challenge” category that are 50% and above are highlighted in blue to indicate an area of challenge. Cells in the “not 
completed” category that are 10% and above are highlighted in orange to indicate a low level of activity completion. 


Tiered Fidelity Inventory 


The Tiered Fidelity Inventory (TFI) examines how schools are implementing PBIS across all three tiers of 
implementation. The TFI is designed to be taken three times per year while the school is in training. 
Schools will have one TFI completed where an external evaluator visits their school and completes a 
“walk-through” to interview staff and students in addition to completing the regular coaching session 
component. Currently, only Cohort 11 schools will be completing the TFI and will do so at each of the 


three trainings this year. 
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Most schools (n=47) in Cohort 11 completed a TFI at fall training this year. The average score for all 
schools in each region for Tier | was 23 percent. In order to be implementing PBIS with fidelity, a school 
needs to score 80 percent or better on each tier. Below are average TFI scores for Tier 1 separated by 
region (Figure 9). 


9. Average Tier 1 TFI scores by region—Cohort 11 


27% ; 
24% 23% 


15% 


Metro (n=18) = North (n=16) = South (n=13) Total average 
across all 
regions (n=47) 


School teams in Cohort 11 that completed the Tiered Fidelity Inventory (TFD at training and were asked to 
identify areas of strengths and challenges based on their outcomes. Many school teams in all regions 
reported that “establishing commitment” to be a strength for their team. All schools in the south region, 
most schools in the north region, and slightly less schools in the metro region indicated that “establish and 
maintain team” is strength. Nearly two-thirds of schools in the North region state that classroom behavior 
support systems are “a challenge.” See figure 10. 


10. Strengths and challenges of the TFI—Cohort 11 


Cohort 11 
Activity Metro North South 
Establish commitment 
It's a strength 68% 86% 77% 
It's a challenge 21% 14% 8% 


Establish and maintain team 


It's a strength 79% 93% 100% 
It's a challenge 16% 1% 0% 
Have not started 5% 0% 0% 
Self-assessment 
It's a strength 42% 43% 62% 
It's a challenge 32% 36% 15% 


Have not started 


Establish school-wide expectations: prevention systems 
It's a strength 32% 50% 38% 
It's a challenge 37% 43% 31% 


Havenotstared 82 
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10. Strengths and challenges of the TFI—Cohort 11 (continued) 


Cohort 11 
Activity Metro North South 
Classroom behavior support systems 
It's a strength 5% 14% 8% 


It's a challenge 53% 64% 23% 
Have not started 2% 21% 694 
Establish information system 


tZ /0 


It's a strength 11% 7% 8% 

It's a challenge 47% 57% 38% 

Have not started 42% 36' 54% 

Build capacity for function-based support 


It's a strength 11% 14% 15% 
It's a challenge 37% 43% 8% 


53% 


Have not started 


Note: Cells in the “it’s a strength” category that are 80% and above are highlighted in green to indicate an area of strength. Cells 
in the “it’s a challenge” category that are 50% and above are highlighted in blue to indicate an area of challenge. Cells in the “not 
completed” category that are 10% and above are highlighted in orange to indicate a low level of activity completion. 


Self-Assessment Survey 


The Self-Assessment Survey, or SAS, examines the status and need for improvement of four behavior 
support systems: school-wide, non-classroom, classroom, and individual students. A summary of the 
survey results is used to develop an action plan for implementing and sustaining Positive Behavioral 
Interventions and Supports (PBIS) systems throughout the school. For Cohort 11 schools, the SAS needs 
to be completed once in the fall and once in the spring to be on track with MDE’s data calendar. The 
majority of schools in Cohort 11 have not yet completed the SAS (n=45), however 11 schools have 
completed a SAS this fall. One school from Cohort 10 has completed the SAS; however, the SAS only 
needs to be completed once during the school year for schools in the second year of PBIS training. 


Behavioral data system 


The presence of a behavioral data system allows for schools to track office discipline referral (ODR) data 
and use this for data-based decision-making. The data collected in these systems are used for improving 
school-wide behavior support. Schools should have an established behavior system in place by winter 
training in their first year. About the same number of schools in Cohort 10 and Cohort 11 use SWIS. 
Slightly more schools in Cohort 11 use a data system other than SWIS compared with Cohort 10 schools. 
One school from each cohort is identified as having “unknown or none” listed as their data system. See 
Figure 11. 
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11. Behavioral data system used, Cohorts 10 & 11 


34 35 


Cohort 10 (N=52) Cohort 11 (N=56) 
= SWIS only 
= SWIS and some other data system 
= Non-SWIS 
Unknown/None 


Issues to consider and recommendations 


Based on the results described above, there are a few things the Minnesota PBIS Statewide Leadership Team, 
including the Minnesota Department of Education (MDE) and the Regional Implementation Partners (RIPs) 
should consider when planning future trainings. RIPs should also pay attention to areas where their 
particular region or a specific Cohort within their region may be rating aspects of the training less favorably 
to determine possible causes and solutions for these specific problem areas. 


Training format and content. School teams generally expressed satisfaction with the training and provided high 
ratings on many aspects of the training. When they were asked to indicate the degree to which they felt the 
content of specific sessions in the training will help their team successfully implement PBIS in their school, 
a majority of school teams frequently rated the sessions as helpful. The majority of school teams in both 
cohorts also believe that the training was a positive, worthwhile experience. 


Schools in Cohort 10 found “Escalating Behavior’ to be the most useful content area for training while 
schools in Cohort 11 found the “Eight Step Planning Process,” “Establishing Buy-in,” “Mission and 
Behavior Purpose,” and “Identifying 3-5 Expectations” as “vitally” or “very useful” to their PBIS 
implementation efforts. 


Additionally, when asked in the open-ended format to provide their opinions on how to improve PBIS 
training, many teams from both cohorts (n=22) stated that work time and action planning were the most 
useful for their teams. This feedback is provided after every training on the evaluations. One option to 
address this feedback could be that the SLT and the trainers could host a workshop day for PBIS school 
teams where the teams come with the expectation that they will work on PBIS things like data, action 
planning, and other activities with assistance from the RIPs and/or trainers. MRIP has already hosted two 
separate “Data Days” for schools out of training—perhaps schools in training could be included in these 
days. Otherwise, the SLT could explore an option to have this day built onto training or it could occur at a 
different time of year. 


Other feedback on how PBIS training could be improved included the need for more concrete, in-depth 
examples on how to implement PBIS (n=9) and having the materials (like the PBIS manual and other prep 
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work) provided to them both at training and ahead of time so they could come to training prepared (n=6). 
See below for some of their comments: 


“Having a general lesson plan outline for teaching basic social skills would be very helpful. It would be great to 
have a starting point to go from and develop lessons off of ideas that are already there.” 
—Cohort 10 team, North region 


‘At this point in our training, we would like to receive more in-depth resources and strategies to really impact our 
staff and students. This felt like a review of information from last year.’—Cohort 10 team, North region 


“We would like handouts available or computers to use.’-—Cohort 11 team, South region 


“We would like real life videos on Tier 2 interventions using high school examples. It seems like more [examples] 
are geared toward elementary/middle schools.’—Cohort 10 team, Metro region 


“We would like more visuals, more materials, and more examples how to roll out PBIS.” 
—Cohort 11 team, Metro region 


The SLT, RIPs, an trainers may need to continue to make a more concerted effort, particularly at the very 
first training session (and possibly through improved communications with schools prior to the start of the 
training sequence) to help school teams understand how the material they are learning at training can and 
should be applied once they get back to school. MDE and the RIPs may also want to consider revising 
training materials based on training survey feedback to include concrete examples of PBIS for schools at 
all grade levels. 


Activities and materials. The majority of teams in all regions in Cohort 10 and the North region in Cohort 11 
believed the resources, materials, and teaching aids were helpful. However, one-quarter of teams from the 
Metro region (26%) and South region (23%) disagreed that these materials were helpful. Considering that 
school teams also commented in the open-ended format that they would like more concrete, in-depth 
examples of PBIS and having more training materials provided to them (see above section for open-ended 
quotes and responses), MDE and the RIPs should consider ways of improving resources, activities, and 
teaching aides that are provided during training. Because these resources will assist schools in implementing 
concrete aspects of PBIS once they are back at school, it is likely that additional efforts here will contribute 
to higher PBIS implementation fidelity measures. 


Individual trainer feedback. For this training survey, Wilder Research attempted to collect data on individual 
trainers and/or their topics to try to make the survey tailored to regions and their trainings and also to inform 
trainers of any areas they might want to change for their trainings. However, the open-ended question 
design which allowed school team members to enter in any trainers or topic area that they wished was not 
uniform and could not be analyzed for the purposes of this report. Therefore, these results are not reported 
here. Wilder will work with MDE and the RIPs to re-design the training survey questions to ensure that they 
are uniform so analysis is possible and the questions produce meaningful feedback for trainers. 


Use of data. Most schools in Cohort 10 and 11 completed the TIC or TFI at training or within the weeks 
prior to training. A majority of Cohort 10 schools fully or partially updated the school’s PBIS Action Plan. 
A majority of Cohort 11 schools partially updated their Action Plan. The majority of Cohort 10 and 11 
teams did not review their ODR data, nor did they research or implement Tier 2 or Tier 3 interventions. 
Given that school was not in session, this is to be expected. However, if these types of activities are 
expected to be completed prior to training, the RIPs and trainers should emphasize this to new and returning 
PBIS teams. 
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Also, it is worth noting that 45 teams in Cohort 11 have not taken the SAS yet this fall. Per MDE’s data 
calendar, Cohort 11 schools are supposed to complete the SAS twice per year. The RIPs and trainers should 
continue to emphasize the importance of the SAS in PBIS implementation fidelity. 


The RIPs and MDE should continue to encourage teams to plan around and review outcome data to ensure 
this key PBIS activity of using data for decisions continues to increase. In addition, the RIPs and MDE 
should continue to provide technical assistance in the areas that are deemed to be challenges and remind 
school teams to view their data before attending fall training. Specifically, MDE, the RIPs, and trainers 
should provide suggestions of when and how to research Tier 2 or Tier 3 interventions and strategies 
schools could use to do so. We also recommend that MDE and the RIPs continue to ensure time is set aside 
at training and that technical assistance and data review is provided to schools in both cohorts to review 


their outcome data. 
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For more information 
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